Sparse Representations of Positive Functions via First- and Second-Order Pseudo-Mirror Descent

We consider expected risk minimization problems when the range of estimator is required to be nonnegative, motivated by settings maximum likelihood estimation (MLE) and trajectory optimization. To facilitate nonlinear interpolation, we hypothesize that search space a Reproducing Kernel Hilbert Space (RKHS). develop first second-order variants stochastic mirror descent employing (i) \emph{pseudo-gradients} (ii) complexity-reducing projections. Compressive projection in first-order scheme executed via kernel orthogonal matching pursuit (KOMP), which overcomes fact vanilla RKHS parameterization grows unbounded with iteration index setting. Moreover, pseudo-gradients are needed gradient estimates for cost only computable up some numerical error, arise in, e.g., integral approximations. Under constant step-size compression budget, establish tradeoffs between radius convergence sub-optimality budget parameter, as well non-asymptotic bounds on model complexity. refine solution's precision, extension employs recursively averaged pseudo-gradient outer-products approximate Hessian inverse, whose mean established under an additional eigenvalue decay condition optimal element, unique this work. Experiments demonstrate favorable performance inhomogeneous Poisson Process intensity practice.